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QUESTIONS 


1. "Dr. LeVois, you have reviewed the statistical approach 
that EPA used in its ETS report, and you have heard the criticisms 
and defenses of that approach that have been offered today. You 
have had experience in the application of epidemiologic techniques 
to regulatory problems at another federal agency, the Centers for 
Disease Control. How does what the EPA did compare with what other 
agencies do? Is the EPA an aberration, or does the problem of 
policy-based science occur throughout the public health 
establishment?" 

2. "As a non-scientist, I want to get an understanding of 
just what it means to shift the standard for statistical 
significance the way that EPA did. I don't know much about 
epidemiology, but I do know something about political polls. 

(a) It is true, isn’t it, that by doubling the possibility of 
error from 5 percent to 10 percent in the ETS studies the EPA 
was doing essentially the same thing as doubling the margin of 
error in a political poll from the typical plus or minus 5 
points to plus or minus 10 points?" 
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(b) I certainly wouldn't want to pay for a political poll that 
predicted results that could swing 10 points either way. What 
about health studies with such wide margins of error? Are 
they worth anything?" 

ANSWERS 

I hope to explain why I do not believe that the EPA has used 
statistics appropriately in their ETS risk assessment. My answers 
to Congressman Emerson's questions can be prefaced by the comments 
of two recognized authorities in the field of applied biostatistics 
concerning the use of what scientists refer to as "one-tailed" 
versus "two-tailed" tests of statistical significance; 

J.L. Fleiss - "By performing a one-tailed test, [the researcher] 

is ruling out as unimportant the possible inference 
that p 2 [rate of disease in the exposed group] is 
significantly greater than p 2 [rate of disease in 
the unexposed group]." 

Page 20. 

"...a one-tailed test is called for only when the 
investigator is not interested in a difference in 
the reverse direction from that hypothesized.” 

Page 21. 
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P. Armitage - 
G. Berry 


"If, however, the investigator intends to report 
his results to his professional colleagues, he is 
ethically bound to perform a two-tailed test." 

Page 21. 

Joseph L. Fleiss. Statistical Methods for 
Rates and Proportions . New York: John 
Wiley & Sons, 1973. 


"The critical value for a one-sided test at level P 
is therefore the same as that for a two-sided test 
at level 2 P. In a sense the distinction is 
semantic. On the other hand there is often a 
temptation to use one-sided tests rather than two- 
sided tests because the probability level is lower, 
and therefore the apparent significance is greater. 
A decision to use a one-sided test should never be 
made after looking at the data and observing the 
direction of the departure. Before the data are 
examined one should decide to use a one-sided test 
only if it is quite certain that departures in one 
particular direction will always be ascribed to 
chance, and therefore regarded as nonsignificant 
however large they are. This situation rarely 
arises in practice, and it will be safe to assume 
that significance tests should almost always be 
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two-sided." 


Page 98. 


P. Armitage and G. Berry. Statistical Methods 
in Medical Research . 2nd ed. Oxford, England: 
Blackwell Scientific Publications Ltd, 1987. 

The EPA has not used an appropriate significance test for 
their meta-analysis of data from selected epidemiologic studies on 
the relationship of spousal smoking and lung cancer (the central 
analysis used by EPA to estimate an ETS/lung cancer risk), nor has 
the EPA adequately addressed fundamental problems in the data used 
for that analysis. 

The first draft of the EPA’s ETS risk assessment used a 
standard two-sided significance test for the main spousal smoking 
analysis. Had the same two-sided significance test been used in 
the revised EPA risk assessment the result would not have been 
statistically significant. In their final risk assessment the EPA 
shifted to a one-sided statistical test. This shift suggests that 
the EPA decided to use a one-sided significance test after 
reviewing the data and observing that a two-sided test did not 
achieve statistical significance. This is exactly the error that 
Armitage and Berry warn against. 

A test of statistical significance is only an aid to decision¬ 
making. Such tests help to describe the level of confidence one 
has that a given result is not due to chance alone. The 95% or 99% 
confidence levels for significance tests are standards used by 
researchers both in federal agencies and the scientific community 
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in general, in both epidemiology and risk assessment. The 90% 
confidence level used in EPA's ETS risk assessment would not be 
accepted by that Agency if it were, for example, submitted by a 
manufacturer as part of an application for an EPA-regulated 
pesticide. A one-in-ten error rate is simply not good enough, and 
should not have been used in the EPA's ETS risk assessment. 

Prior to release of the EPA document new spousal smoking data 
were published in two studies, including the largest U.S. study, 
that brought the pooled lung cancer risk estimate down even 
further, to a level that would not be significant at even the 90% 
confidence level. Such statistical instability plainly contradicts 
EPA's claim of "medium to high" confidence in their ETS risk 
estimate, and underscores the need for higher scientific standards 
than they have employed. 

Tests of statistical significance are valid aids in decision¬ 
making only if the underlying data are unbiased. The error EPA 
made by employing a one-sided, 90% significance test should not 
obscure a more basic and important error. The EPA has pooled 
results from spousal smoking studies that are not adequate to 
answer questions about possible health effects of ETS exposure. 
This is because spousal smoking is a biased proxy for ETS exposure. 
Spouses of smokers tend to have greater exposure to other lung 
cancer risk factors than do spouses of nonsmokers. The EPA 
attributes all of the reported spousal smoking-associated lung 
cancer risk to ETS exposure, even though it is clear that spouses 
of smokers tend to have poorer diets, lower socioeconomic status, 
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and increased occupational exposures, and are more likely to have 
a history of prior pulmonary illness than are the spouses of 
nonsmokers. The EPA's ETS meta-analysis simply pools biases 
contained in the individual studies to produce a biased result. 

Biases introduced when spousal smoking is employed as a proxy 
for ETS exposure could easily be responsible for the very weak 
reported pooled spousal smoking / lung cancer risk estimate. It is 
noteworthy that meta-analysis of results of studies that have 
looked at ETS exposure in the workplace (as opposed to using 
spousal smoking as the ETS exposure definition) shows no increase 
in ETS related lung cancer risk. This important discrepancy 
undermines both the validity of the spousal smoking proxy and the 
inference that ETS exposure causes lung cancer. 
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